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ON THE QUASICONVEXITY OF THE FIXED SUBGROUP 
OF ENDOMORPHISMS OF RELATIVELY HYPERBOLIC 

GROUPS. 

V. METAFTSIS AND M. SYKIOTIS 



o, 

Abstract. We prove that for any finitely generated relatively hyperbolic 
5— i ■ group G and any symmetric endomorphism cj> of G with relatively quasicon- 

ft' vex image, Fix^) is relatively quasiconvex subgroup of G. 

in 



1. Introduction 



For a group G and an endomorphism <fi, we denote by Fix0 the subgroup of 
G that is fixed pointwise by 4>. The subgroup Fixcj) has been studied by various 
authors and for various groups G (see [15] and references therein). 

Relatively hyperbolic groups are groups that admit a geometrically finite 
action on a proper hyperbolic space. It is a large class of groups introduced 
by Gromov in its seminal paper [5] and it has attracted exceptional interest 
with very important results. The novice reader should consult the work of 
Bowditch [1, 2], the work of Tukia [14, 13] and of Osin [10] and Gerasimov [3] 
for different approaches. The role of relatively quasiconvex subgroups in the 
theory of relatively hyperbolic goups is as much central as that of quasiconvex 
■^J- , subgroups in hyperbolic groups. Many authors have given different formulations 

■^+ | for this idea, all of which turned out to be equivalent in the standard setup of 

countable relatively hyperbolic groups, as shown by Hruska [6]. 

Neumann [9] proved that the fixed subgroup of any automorphism of a hy- 
perbolic group is quasiconvex. Recently, Minasyan and Osin [7] generalized 
Neumann's result by proving that if G is a finitely generated relatively hyper- 
s' bolic group, then Fix</> is relatively quasiconvex in the case of an automorphism 
4> of G that permutes the maximal parabolic subgroups. 

If G is a relatively hyperbolic groups, then it is equivalent to say that G 
admits a geometrically finite convergence group action on a compact, metrizable 
space X. Following [12], we call an endomorphism eft of G symmetric if there 
exists a continuous map <j) : X — y X such that 4>(gx) = 4>{g)(j){x). The results 
in [12] can be used to obtain classes of relatively hyperbolic groups in which 
the fixed subgroup of any symmetric endomorphism is relatively quasiconvex. 
Therefore, it is very natural to look for generalizations of Minasyan and Osin's 
result to endomorphisms of relatively hyperbolic groups. 
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In the present work we show that for any symmetric endomorphism <f> of a 
geometrically finite group G the fixed subgroup Fix</> is geometrically finite. 
Notice that if G is a finitely generated geometrically finite group, and <p is any 
automorphism which permutes maximal parabolic subgroups, then it follows 
from [2, Theorem 9.4] that (f> induces a homeomorphism <j> of AG such that 
4»{gx) = (j)(g)(p(x). Hence Fix</> is relatively quasiconvex and as an immediate 
consequence, we retrieve the result by Minasyan and Osin [7]. Notice also that 
using free products of locally quasiconvex relatively hyperbolic groups with 
finite groups, we can easily construct symmetric endomorphisms of relatively 
hyperbolic groups which are not even monomorphisms. 

Our approach differs than this in [7] ; it builds heavily on the work of Tukia 
in [14] and is based on the action of G on its Gromov boundary rather than its 
action on a hyperbolic model space. 

2. Convergence groups and equivariant maps 

We begin by recalling some facts about convergence groups. The reader 
could find more in the articles of Bowditch [2] (and in [1]), Tukia [13] and 
Gerasimov [3]. Suppose that X is a compact metric space and G a group that 
acts by homeomorphisms on X. We say that G is a convergence group if the 
induced action on the space T of distinct triples of X is properly discontinuous. 
We write AG C X for the limit set of G, that is the unique minimal closed 
non-empty G-invariant subset of X. We may also assume that the action is 
minimal, in the sense that AG = X. So from now on, we identify X with AG 
and assume that X contains at least three points. Such an action is called 
non-elementary. Otherwise the action is elementary and in that case there is 
very little information we can get for G. For example, if AG contains only one 
point then G can be any countable group. 

A subgroup of G is parabolic if it is infinite, fixes some point of X and contains 
no loxodromics. The fixed point is unique and called parabolic. The stabilizer 
of a parabolic point p is a parabolic subgroup H of G. If H acts cocompactly 
onI\ {p}, then we say that p is a bounded parabolic point. A point p £ X is a 
conical limit point if there exist a sequence gi € G and points a ^ b in X such 
that gip — > a but g{X —?■ b for any other x € X. Conical limit points cannot be 
parabolic ([14, Theorem 3A]). 

The (convergence) action of G is geometrically finite (or cusp-uniform) if 
every point of X is either conical or bounded parabolic. In that case G is a rel- 
atively hyperbolic group and there are finitely many orbits of bounded parabolic 
points [14]. The set of maximal parabolic subgroups is the peripheral structure 
of G. A subgroup H of G is called relatively quasiconvex if the induced conver- 
gence action of H on its limit set is geometrically finite. We refer the reader to 
[6] for equivalent characterizations of relative hyperbolicity. 

Following Tukia we set T = TU X and define the topology of T so that T is 
an open subset of T and the distance between z = (zi,Z2,Zs) € T and x € X is 

d(z, x) = the second largest of d(zi, x), i = 1, 2, 3 

with the assumption that if two of d(zi,x) coincide then d(z, x) is this distance. 
The above distance defines a metric that extends the metric of X. Notice 
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however, that d is not defined if both x, z £ T (see [14, page 77]). Thus we can 
define an e-neighbourhood of x € X to be the elements {z G T \ d(z,x) < e}. 
Then T is compact and the action of G on X and T induces a convergence 
action on T, proper discontinuous on T. 

Now let be a symmetric endomorphism of G, i.e. there exists a G-equivariant 
continuous map : X — > X, in the sense that (j>(gx) = <p(g)<p(x). The map 
(j) can be extended naturally to a continuous map <fi : T — > T such that 
(j)(g(xi,X2,Xs)) = <f)(g)((f)(xi),(f)(x2),<f>(x3)). The fact that cp is continuous is 
an immediate consequence of the fact that T inherits the product topology of 
X. The G-equivariance of is a consequence of the definition of the action of 
G on T. This extension allow us to define a G-equivariant extension eft : T — > T 
with _ 

nX) ~ \ 0(x) if x € T 

Lemma 1. Under the above hypotheses, <j) is continuous and AFix0 C Fix</> = 
{x £ X \ ~4>(x) = x)}. 

Proof. To show that (f> is continuous, it suffices to prove that (j) is continuous 
in each x € X, since T is an open subset of T. So let x G X and (p(x) = (f>(x) 
its image under (p. Let also e > and V((/>(x),s) = {z 6 T | <i(z,x) < e} an e- 
neighbourhood of <j){x). Since (f> is continuous, we have that for every e > there 
is a 5 > such that the 0-image of the (5-neighbourhood of x in X is contained 
in the e-neighbourhood of <j>{x) in X. Let V(x,5) = {z G T | d(z,x) < S}. 
Then ^(y(x, <5)) C y(^(x),e). Indeed, for any z = (zi,Z2, z$) € V(x,S) P\T, 
at least two of the distances {d(zi,x),d(z2,x),d(z3,x)} are less than 5. Hence, 
continuity of (f> implies that at least two of three distances {d((f)(zi), ^>(x))}j = i ; 2,3 
are less than e. On the other hand, by the continuity of (f>, all elements in 
V(x, (5)fll are also mapped by <\> into V((f)(x), e). 

Now if x £ Fix0, then for every g £ Fix^ we have 4>(gx) = 4>(g)(f>(x) = gx 
and so Fix<^> is Fix^>- invariant. Moreover, for any sequence (x n ) of elements of 
Fix(f> with x n —?• x € X we have that <j>{x n ) — > (f>(x) by continuity and hence 
x £ Fixcf). Consequently, Fix0 is a closed Fix^-invariant subset of X. Hence, 
AFix0 C Fix0, by the definition of AFix<^>. D 

3. The main result 

In the present section we show that if G is a non-elementary convergence 
group, acting geometrically finite on a compact metric space X and (ft a sym- 
metric endomorphism of G, then AFix0 consists only of conical limit points 
and bounded parabolic points. In fact, we shall show that the quotient of 
T' = (AFix^) 3 \ A, where A is the diagonal, under the action of AFix^> is the 
union of a compact set and a finite number of AFix</>-quotients of cusp neigh- 
bourhoods of bounded parabolic points of AFix</>. For notational simplicity, let 
us denote T' = T'U AFix^ C T. 

Assume that p is a bounded parabolic point of X. Then G p acts properly 
discontinuously and cocompactly on X \ {p} ([14, Theorem 3A]). Let C be a 
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compact set such that G p C = X \ {p} and W a compact neighbourhood of C in 
T not containing p. Define a cusp neighbourhood of p to be an open subset of 
T of the form T \ G P W. A cusp neighbourhood U p of p is G p -invariant. A cusp 
neighbourhood is called precisely invariant if gU p = U p for every g G G p and 
gU p PI Up = if g G G \ G p . Lemma 3F in [14] proves the existence of precisely 
invariant cusp neighbourhoods for every parabolic point of G. Let c\tU p denote 
the closure in T of U p . Then, by making U p small enough we may assume that 
c\tU p is also precisely invariant (again, see [14]). 

In [14, Theorem IB] it is shown that G is relatively hyperbolic if and only if 
T/G is the union of a compact set and a finite number of G-quotients of cusp 
neighbourhoods of bounded parabolic points. We shall make extensive use of 
this result in what follows. 

Lemma 2. With the above hypotheses, T' = (Fix</>)P' U (\J(U qi nT')) for some 
compact subset P' C T" and a collection {q{\ of bounded parabolic points of G 
such that {q{\ C AFix^. 

Proof. From hypotheses we know that T = GP U (|J U Pi ) where P is a compact 
subset of T and U Pi are cusp neighbourhoods of bounded parabolic points pi 
in T. Without loss of generality we may assume that U Pi n T' = whenever 
Pi is not a limit point of AFix</>. The lemma will be proved by constructing a 
compact subset P' of T" and a cusp neighborhood U' for each parabolic point 
Pi which is also a limit point of AFix</> such that GP D T" = (AFix0)P' and 

u Pi = u Pi nT>. 

Let G = \J i€l (Fix.(f))gi for some right coset representative system {gi}i£i of 
Fix</> in G. Since <p is a continuous function, we have that (f>(P) is a compact 
subset of T. Define Pa = P U <p(P)- We claim that there are finitely many gi 
such that giPPiT' ^ 0. To see that, let x 6 P such that g>jX € T'. So g{X G Fix0 
and so 4>(gix) = giX. Hence, <fi(gi)(f)(x) = g. L x or equivalently <?~ 4>(gi)4>(x) = x. 
But <^(x) € 4>{P) and so g>~ <P{9i) & S where 

S = {g£G\gP 4 ,nP (t>7 L 0}. 

But G acts properly discontinuously on T and so S is a finite set. Moreover, 
if g~ x (j){gi) = g^cpigj) then gjg' 1 £ Fix^ and so (Fix^)^ = (Fix0)#,-, thus 
^j = <7j. This proves the claim. 

Let gi, . . . , g k G {#i}i e / such that 5jP nT'^0 and 

P' = ( 9 iPu...u ft P)nT'. 

By its choice, P' is a compact subset of T' . This is the case since T is closed 
in T and ( 5i P U . . . U g k P) nT' = (51 P U . . . U # fc P) n t'. We show that 

(Fix^)P' = GP n T'. 

It is obvious that (Fix0)P ; C GPCiT'. Let y G GPDT'. Then y = gx for some 
x G P. On the other hand, g = /i(/a f° r some h G Fix0, and so h~ 1 y = g\x. 
Therefore, g\x G g\P and /i _1 y G X" since T" is invariant under the action of 
Fix</>. Hence, <7a^ G (/aP H T' and so 5a G {51, . . . ,gk}- Thus, y = /if ix which 
implies that y G (Fix0)P'. 
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Next, using similar arguments, we show that the same properties are true for 
the cusp neighbourhoods of every bounded parabolic point of G that is also a 
point of AFix0. Indeed, let p be a bounded parabolic point of AG such that 
p G AFixcfr. Choose G a compact subset of AG\{p} such that G p C = AG\{p}. 
Then take a compact neighbourhood W of C in T \ {p}. The open subset 
Up = T\ G P W is a cusp neighbourhood of p. 

Let H = G p fl Fix</> and assume that G p = \J ieI Hq h for some right coset 
representative system {qi}i£i- Since C is compact and (j> a continuous function 
of X, again, <p{C) is a compact subset of A. Define C^ = GU^(G). To show that 
there are finitely many qi such that qiC n AFix0 \ {p} ^ 0, let x G C such that 
qiX £ AFix0 \ {p}. Then qiX £ Fix^ and so (p(qix) = qiX, i.e. <p(qi)<p(x) = qiX 
or q~ (f)(qi)4>(x) = x. But 4>(x) £ <j>{C) and so 

9 rV(ft) g 5i = { 5 g G p I ^ nQ/ 0}. 

But G p acts properly discontinuously on AG\{p} ([14, Theorem 3A]) hence S\ 
is a finite set. Moreover, if q~ (f)(qi) = qj 4>(qj) then qjq~ G Fix0 n G p and 
so i?% = Hqj so g» = qj. Let q±,. . . ,q r be the elements of {^jig/ for which 
qiC n AG \ {p} / for i = 1, . . . , r and denote C" the set 

C = {qiC U . . . U q r C) n AFix0 \ {p}. 

It is obvious that C is a compact subset of AFix0, not containing p since AFix<j) 
is a closed subset of AG and 

(qiC U . . . U q r C) n AFix</> \ {p} = {q x C U . . . U g r G) n AFix0. 

Again, 

HC' = GpC n AFixcf) \ {p} = AFix^ \ {p}. 

Indeed, let y G G p C D AFix^> \ {p}. Then y = gx for some x G G and some 
5 G G p . Again g = hq\ for h G (G p fl Fixtfi) and /i _1 y = q\x. So g,\x G 
g A GnAFix0\{p}. Hence q\ G {gi, . . . , q r } and y = hqiX G (G p nFix0)G' = HC'. 
Hence, C is a compact subset of AFix^ \ {p} such that (G p nFix</>)G' = HC' = 
AFix<p\{p}. 

Now we use the compact set C to construct the cusp neighborhood U' as 
follows. Let M = q\W U . . . U q r W . Then M is a compact subset of T that 
belongs entirely into T \ {p}. We claim again that there are finitely many 
k G {%}ie/ such that t { M dT' \ {p} / 0. Define M^ = M U </>(M). M is 
compact, since eft is a continuous function. If x G M with £jx £ T \ {p}, then 
ijX G Fix</> and <j){Ux) = Ux, i.e. £~ <p{ti)<j){x) = x. But 0(x) G 4>(M) and so 
£~ (f)(ti) £ {g £ G p \ gM^ n M^ 7^ 0} which is a finite set by the discontinuity of 
the action of G p on T \ {p} ([14, Theorem 3A]). Moreover, t~ 4>(ti) = t~ 4>(tj) 
implies that tjt~ £ G p r\Fix<p = H and so HU = Htj, i.e. t% = tj. Let t\, . . . ,t n 
the elements of {qi}iei for which ijM fl T \ {p} 7^ 0. 

Let 

W' = (hM U . . . U t n M) nf'\ {p}. 
It is apparent that 

W' = (nW U . . . U r m W) n T' \ {p} 
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with t{ G {<li}i£i and r,,W are the only "copies" of W for which we might 
have nW r\T'\ {p} ^ 0. Define W" = (viW U . . . U v^W) C\T'\ {p} where 
Vi€{n,..., r m } and v t W D T' \ {p} + 0. 
We have 

HW" = G p w n T' \ {p}. 

For y G G P W (IT \ {p} we have g G G p and x £ W such that y = gx. Again, 

# = hq\ with h £ H and /i _1 y = (/a^- So (/a^ G qaH^ HT \ {p} and so 
q\ G {ui, . . . , i^} and y = hr { x G flW". 

Now {fi,...,^} = {gi,...,g r .}U{g r+ i,...,g a }. Since qfi n AFix^ \ {p} = 
for all i G {g r+ i, . . . , (7 a } but q{W fl T \ {p} ^ 0, VF is a compact subset and T 
is closed in T, we have that qiW C\T \ {p} is a compact subset of T". Hence, 
we may consider W" as a compact neighbourhood of C in T not containing p. 

Take 

Up = T'\ (G p n Fix0)W". 
Then, by its construction, U' is a cusp neighbourhood of p G T such that 

u p = u p nr. ' a 

The following is an immediate consequence of the above proof. 



Corollary 1. Every bounded parabolic point of G that is also an element in 
AFixcf) is a bounded parabolic point ofFix<fi. 

Lemma 3. With the above hypotheses, T'/Fix</> is the union of a compact subset 
P' and a finite number of Fix<fi-quotients of cusp neighbourhoods of bounded 
parabolic points of AFixcf). 

Proof. Assume that {U Pi } are chosen such that c\TU Pi are precisely invariant. 
Assume further that T'/Fix0 contains an infinite number of Fix^>-quotients 
of cusp neighbourhoods {U'} as constructed in Lemma 2. Since every U' is 
induced by some U p , there is a bounded parabolic point p G AFix^> and infinitely 
many gi G Fixcj) such that giU p /Fixcfi are different in T' /Fixcfi. This means that 
gi is a sequence of discrete elements of Fixcft and so there is a subsequence, 
denoted again gi, such that gip — > b for some b G X. But by Lemma 3E [14], 
giUp accumulate, a contradiction to the precisely invariant choice of {U Pi }. □ 

Now we are able to prove our main theorem. 

Theorem 1. Let G be geometrically finite group and <j) a symmetric endomor- 
phism of G. Then Fix0 is geometrically finite. 

Proof. If AFixcj) = 0, then Fixcj) is finite and hence quasiconvex in a trivial way. 
If AFixcj) has one point then Fixcf) is parabolic and is relatively quasiconvex. If 
Fixcp has two points then Fixcj) is cyclic-by-finite, and is contained as a subgroup 
of finite index into a unique maximal cyclic-by-finite elementary subgroup (see 
[13]). Hence, Fixcj) is quasiconvex in G by the result of Osin [11, Corollary 1.7]. 
So we may assume that AFix</> has at least three points. Then the result is 
an immediate consequence of Lemmas 2 and 3. □ 
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4. Applications 

Let G be a finitely generated relatively hyperbolic group and <fr a monomor- 
phism of G which maps maximal parabolic subgroups into conjugates of them- 
selves, i.e. for each maximal parabolic subgroup G p there is a maximal parabolic 
subgroup G q such that (/)(Gp) C G q . In the special case where <f> is an auto- 
morphism permuting maximal parabolic subgroups, the result of Bowditch [2, 
Theorem 9.4] shows that (j) is symmetric. In general, in order to show that 
<fi is symmetric we assume further that its image is a relatively quasiconvex 
subgroup of G. What shall need the following result. 

Theorem 2 ([8, Theorem 1.1]). Let G be a countable group and X, Y compact 
metrizable spaces endowed with geometrically finite actions of G. If each maxi- 
mal parabolic subgroup with respect to X is a parabolic subgroup with respect to 
Y , then there exists a continuous map tt : X —?■ Y such that ir(gx) = gir(x) for 
each g £ G and x £ X. 

Theorem 3. Let G be a finitely generated group acting minimally and geomet- 
rically finite on a compact metrizable space X. Suppose that <f> is a monomor- 
phism of G such that (j){G) is a geometrically finite subgroup of G. If <fi maps 
maximal parabolic subgroups into conjugates of themselves, then 4> is symmetric. 
In particular, Fix</> is geometrically finite. 

Proof. We consider the action of G on A<fi(G) giving by g * x = <p(g)x. Since (j) 
is monomorphism and 4>(G) is geometrically finite, it follows that the twisted 
action of G on A(f>(G) is geometrically finite. Moreover, the maximal parabolic 
subgroup corresponding to a parabolic point p & A<p(G) is (j)~ 1 (G p ), where G p 
is the parabolic subgroup of the initial action of G on X = AG. Thus the 
hypotheses of the above theorem are satisfied and we have a continuous map 
■k : AG — > A(f)(G) C AG such that w(gx) = (f)(g)ir(x). This proves that <f> is 
symmetric and therefore Fix0 is geometrically finite. □ 

An immediate consequence of Theorem 3 is the following result by Minasyan 
and Osin. 

Corollary 2 ([7]). Let G be a finitely generated relatively hyperbolic group and 
(j) a symmetric automorphism of G. Then Fix0 is relatively quasiconvex. 

A group G is locally quasiconvex if all its finitely generated subgroups are qua- 
siconvex with respect to some finite generating set of G. Locally quasiconvex 
subgroup are studied in [4]. The family of locally quasiconvex groups contains 
free groups, surface groups, certain Coxeter groups and certain small cancel- 
lation groups. Moreover, as shown in [4] it is closed under free products and 
certain amalgamated free products. Theorem 3 has the following consequence 
concerning locally quasiconvex hyperbolic groups. 

Corollary 3. If G is a locally quasiconvex hyperbolic group, then Fix</> is qua- 
siconvex for any monomorphism (j) of G. 
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